The bug

generate_challenge_id() serializes the request with ensure_ascii=False, but the code that writes
the request parameter into the header does not. For any non-ASCII character the HMAC signs one
string while the header carries another, so a peer cannot recompute the id and rejects the challenge
with invalid-challenge.

Section 5.1.1 of draft-httpauth-payment-00 binds the id to the request as it appears on the wire,
so the two have to be the same bytes.

Sites that had drifted to the ensure_ascii=True default:

• _parsing.py:43 (_b64_encode), reached from format_www_authenticate at :169
• extensions/mcp/types.py:92,190,195

Before

sequenceDiagram
    autonumber
    participant S as pympp server
    participant P as Peer verifier
    Note over S: request contains a non-ASCII character
    S->>S: generate_challenge_id(), ensure_ascii=False
    Note right of S: id = HMAC(raw UTF-8 bytes)
    S->>S: format_www_authenticate(), ensure_ascii=True
    Note right of S: header carries escaped bytes
    S-->>P: 402 with that id, but escaped request
    P->>P: recompute HMAC from the header value
    Note right of P: HMAC(escaped bytes), does not match
    P-->>S: rejected, invalid-challenge

After

sequenceDiagram
    autonumber
    participant S as pympp server
    participant P as Peer verifier
    Note over S: request contains a non-ASCII character
    S->>S: generate_challenge_id() via canonical_json()
    S->>S: format_www_authenticate() via canonical_json()
    Note right of S: one definition, same bytes
    S-->>P: 402 with that id, raw UTF-8 request
    P->>P: recompute HMAC from the header value
    Note right of P: same bytes, matches
    P-->>S: accepted

Why the tests missed it

verify() decodes the echoed request back to a dict and re-serializes it before recomputing the id,
so pympp always agrees with itself. The mismatch only appears against another implementation.

test_unicode_in_description covers generate_challenge_id(), which was already correct. Nothing
covered the header.

The fix

The same serialization rule appeared at nine call sites and four of them had drifted. This routes
all nine through one canonical_json() helper.

Read _parsing.py first, it holds the fix. Then __init__.py, extensions/mcp/types.py and
_body_digest.py, which just call the helper, then the tests.

Commit 1 is the behavior change, commit 2 is the tests. Both pass on their own.

Testing

make lint && make format-check && make test

780 passed, 41 skipped (778 on main, plus the 2 new tests).

To watch the tests fail without the fix, cherry-pick the test commit onto main and run
uv run pytest tests/test_challenge_id.py -k WireForm.

One thing worth a second opinion

The smallest possible fix is four ensure_ascii=False arguments. I used a shared helper instead,
which touches __init__.py and _body_digest.py even though they were already correct. My reasoning
is that nine copies of one rule, four of them drifted, is the actual problem. If you would rather
keep the diff minimal, I will cut it back.

Only non-ASCII requests change. ASCII output is byte-identical, so existing challenge ids are
unaffected, and the ids that do change are ones no other implementation could verify anyway.